Learning Optimal Wavelets from Overcomplete Representations

Efficient and robust representation of signals has been the focus of a number of areas of research. Wavelets represent one such representation scheme that enjoys desirable qualities such as time-frequency localization. Once the Mother wavelet has been selected, other wavelets can be generated as translated and dilated versions of the Mother wavelet in the 1D case. In the 2D case tensor product of two 1D wavelets is the most often used transform. Overcomplete representation of wavelets has proved to be of great advantage, both in sparse coding of complex scenes and multimedia data compression. On the other hand overcompleteness raises a number of technical difficulties for robust computation and systematic generalization of constructions beyond their original application domains. In the following, we concentrate on wavelet decomposition of images, and in particular, the computations are carried out for images of faces. With more effort, video files could be handled with adaptation of our present techniques. In this paper we propose a novel and quite general geometric method to generate overcomplete families that are parameterized by wellbehaved manifolds, in fact Lie groups. The main philosophical motivation for our construction is a statistical adaptation of Felix Klein's Erlanger Program. That is, knowledge about the symmetry groups of objects in a family D (called the data set, such as collection of multimedia data or images) carries much of the key statistical geometric features of D and the distribution of such features for members of D in a statistical sense. We use function theory and the geometric properties of the parameter space to extract sparse representations aiming at optimal use of the local geometric features of members of D. The better the statistical distribution of such geometric features, the more efficient and robust becomes the wavelet representation of members of the data set. Briefly, the theory begins with generating the overcomplete family from the local and global transformations afforded by the data sets, e.g. images. In the second stage, we formulate an "action integral" on the suitable function space associated with the moduli space of wavelets. Finally, variational formulas that minimize "the action" provide the sparse representation of data in the form of special points on the moduli space. We illustrate this method by studying the problem of sparse representation of images of faces in a collection D. In the specific application to compute sparse representation of facial images, we develop a specific learning machine to take advantage of the rather rich statistical regularity of frontal view of face images. Rather than starting with a mother wavelet and searching for an optimal representation of a single image, we start with a set of images and ask for a mother wavelet that gives the optimal representation for this class of images. We choose the Support Vector Machine paradigm as the learning paradigm. Then, our problem is one of estimating an optimal kernel in the Reproducing Kernel Hilbert Space that satisfies the admissibility conditions of wavelets. We show simulations results using a database of human faces using a set of wavelet kernels. We discuss extensions of this approach using a local to global technique for eigenfunction expansion.